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DETAILED ACTION 

Information Disclosure Statement 

1 . The Information Disclosure Statement filed 04 November 2008 fails to comply 
with the provisions of 37 CFR 1 .97, 1 .98 and MPEP § 609 because it does not 
represent prior art, but is only attorney work-product produced for purposes of litigation. 
Accordingly, the information referred to therein is not being considered. 

2. The Information Disclosure Statements filed 22 October 2008, 25 November 
2008, and 09 December 2008 fail to comply with the provisions of 37 CFR 1 .97, 1 .98 
and MPEP § 609 because they do not set forth dates of publication for some of the 
Non-Patent Literature. All Non-Patent Literature must at least have a year of publication 
for it to be considered. Non-Patent Literature not having dates of publication are lined 
through on the Information Disclosure Statements. All remaining prior art on the 
Information Disclosure Statement is being considered unless it is lined through. 

3. The Information Disclosure Statement filed 22 October 2008 fails to comply with 
the provisions of 37 CFR 1 .97, 1 .98 and MPEP § 609 because it is a duplicate of 
another Information Disclosure Statement filed on 22 October 2008. Accordingly, only 
one of the two Information Disclosure Statements filed 22 October 2008 is being 
considered; the other is not being entered. 
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Claim Rejections - 35 USC §112 

4. The following is a quotation of the first paragraph of 35 U.S. C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

5. Claims 14, 56, 72, and 74 are rejected under 35 U.S.C. 112, first paragraph, as 
failing to comply with the written description requirement. The claims contain subject 
matter which was not described in the specification in such a way as to reasonably 
convey to one skilled in the relevant art that the inventors, at the time the application 
was filed, had possession of the claimed invention. 

Claims 14 and 56 contain the limitation of computing compressed MFCC 
coefficients at either the client system or the server computing system "on a connection 
by connection basis", which limitation is not supported by a written description in the 
originally-filed Specification. Applicants' Specification discloses optimizing computation 
of MFCC coefficients on a case-by-case basis, U[0173], and a system-by-system basis, 
10174], of corresponding U.S. Patent Publication 2005/0080625, but not on a 
connection-by-connection basis. The reconfiguration could be performed many times 
during a single connection, or could be performed on a set schedule, e.g., daily. The 
Specification does not disclose that the reconfiguration is performed on a connection by 
connection basis. 

Claims 72 and 74 contain the limitation of "said client system is a portable 
electronics device and a data content representative of speech data values is 
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configured based on a processing ability of such device", which limitation involves new 
matter because it is not supported by the originally-filed Specification. Corresponding 
U.S. Patent Publication 2005/0080625, 1f[0081], briefly discloses a client computing 
system can be as simple as a personal digital assistant or a cell phone, but does not 
disclose configuring speech data values based upon a processing ability of a portable 
electronics device. The Specification, U[0173] to U[0174], discloses partitioning of signal 
processing capabilities between the client and server side, but partitioning of signal 
processing capabilities is not disclosed for a portable electronics device. 

Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

7. Claims 11 to 13, 53 to 55, 71, and 73 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Barclay et al. in view of Lennig et al. 

Concerning independent claims 11 and 53, Barclay etal. discloses a client/server 
speech recognizer, comprising: 

"a first audio signal receiving routine for receiving user speech utterance signals 
representing speech utterances to be recognized during a sequence of speech 
utterance evaluation timeframes, said speech utterances including sentences 
comprised of one or more words" - client system 2 has microphone 10 that accepts 
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audio input, converts a sound into a digital representation by analog to digital 
conversion (ADC), and extracts a set of features that provide the best recognition 
(column 4, line 56 to column 5, Iine10: Figure 1); speech is recognized as words and 
sentences for an utterance, "Show me the weather for Boston" (column 3, lines 6 to 24), 
or "I want to fly from Boston to Denver tomorrow" (column 9, lines 16 to 25); continuous 
speech is divided into discrete transformed segments that facilitate mathematical 
operations (column 2, lines 14 to 16); implicitly, these discrete segments are called 
'frames' ("a sequence of speech utterance evaluation time frames"); 

"a first signal processing routine adapted to generate representative speech data 
values for each speech utterance evaluation time frame during which speech utterance 
signals are received, [said representative speech data values including a set of 
compressed mel-frequency cepstral coefficients (MFCC)]" - client system 2 has front- 
end 12 that extracts a set of features ("representative speech data values"), called 
cepstra, that provide the best recognition, and the features are quantized (column 5, 
lines 2 to 1 1 : Figure 1); 

"a formatting routine for rendering said representative speech data values into a 
transmission format suitable for transmission from the client system over a 
communications channel to a second processing routine executing on the server 
computing system" - dispatcher program of the client 70 sends quantized speech 
features via a communications link to server computer system 80 (column 5, line 48 to 
column 6, line 6: Figure 4); a dispatcher implements transmission by TCP/IP protocol 
(column 6, lines 50 to 53); 
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"wherein said representative speech data values are transmitted continuously 
during said speech utterances within streaming packets and without waiting for silence 
to be detected and/or said speech utterances to be completed" - the dispatcher does 
not wait for completion of logical blocks and the dispatcher itself is designed to quickly 
service the Internet with minimum overhead time (column 6, lines 56 to 59); front end 
streams the quantized data to the dispatcher, where 'stream' is defined as to send 
substantially continuously the data in real-time (column 7, lines 13 to 19); features are 
sent as packets (column 7, lines 48 to 59), implicitly, in TCP/IP protocol, a most likely 
transcription or text is determined until an end-of-speech (EOS) is received (column 6, 
lines 5 to 8); thus, a dispatcher does not wait until an end-of-speech, i.e. silence, is 
detected to initiate transmission of quantized feature data; 

"further wherein said representative speech data values constitute a minimum 
amount of information that can be used by said second processing routine to complete 
accurate recognition of said one or more words and said sentences" - extracted sets of 
features, called cepstra, are those features that provide the best recognition as known 
in the art (column 5, lines 2 to 8); implicitly, cepstra are "a minimum amount of 
information that can be used" for speech recognition because they are a compressed 
form of the most relevant features for speech recognition; 

"and said communication channel further being configured by the machine 
executable program such that grammar related information is sent to the second 
processing routine executing on the server computing system for identifying a grammar 
to be used for recognition of said one or more words from said sentences" - when the 
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client is sending to the server, the client may specify what grammar the speech 
processor should use to recognize the current speech input or provide extra arguments 
over and above the speech inputs (column 8, lines 22 to 28). 

Concerning independent claims 11 and 53, Barclay etal. discloses cepstra as 
representative speech data values, but does not expressly disclose "said representative 
speech data values including a set of compressed mel-frequency cepstral coefficients 
(MFCC)". However, mel-frequency cepstral coefficients (MFCC) are a well known form 
of cepstra in speech recognition. Specifically, Lennig etal. teaches a speech 
recognizer, where the spectral representation of speech is mel-based cepstral and 
dynamic components. (Abstract) Mel-frequency cepstrum may be used as short-term 
spectral representations of primary parameters. (Column 1 , Lines 5 to 18; Column 2, 
Lines 9 to 1 7; Column 2, Lines 43 to 47; Column 6, Lines 1 to 1 1 ) An advantage is that 
mel-based cepstral coefficients can represent a spectral shape and change in spectral 
shape, in addition to changes in loudness or amplitude, so that dynamic parameters can 
result in a reduction of recognition errors over a public switched telephone network of 
about 20%. (Column 2, Lines 42 to 49; Column 6, Lines 1 to 1 1 ) It would have been 
obvious to one having ordinary skill in the art to utilize mel frequency cepstral 
coefficients as taught by Lennig et al. in a client/server speech recognizer of Barclay et 
al. for a purpose of obtaining a reduction of recognition errors over a network by 
representing dynamic parameters for changes in spectral shape and amplitude. 
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Concerning claim 12, Barclay et al. discloses incorporating a dispatcher into a 
browser of a client side application for speech recognition at a server (column 8, lines 
36 to 64: Figure 4). 

Concerning claims 13 and 55, Lennig etal. discloses estimating a magnitude or 
power spectrum for mel-frequency cepstra over a time frame between 2 and 50 mS 
(column 2, lines 56 to 65; column 6, lines 43 to 48); 1 00 frames per second implies 1 
frame = 1/100 second, or 1 x 10~ 2 seconds = 10 x 10" 3 seconds, or about 10 
milliseconds; thus, a 2 millisecond time frame for calculating cepstra corresponds to 
about 500 frames per second; energy is determined for a plurality of twenty channels 
spanning a frequency range from about 100 Hz to about 4000 Hz (column 3, lines 4 to 
11); each channel corresponds to "a corresponding frequency component", and a 
frequency range from 100 Hz to 4000 Hz is equivalent to "an audible speech frequency 
range." 

Concerning claim 54, Barclay et al. discloses real-time speech recognition 
processing and response (Abstract). 

Concerning claims 71 and 73, Barclay etal. discloses a LISTEN button on a 
graphical user interface (GUI) of a client, so that speech recognition starts when a user 
presses a LISTEN button ("in response to a button being pressed on the client system") 
(column 8, lines 48 to 64); when a user clicks on LISTEN, natural language processing 
begins for a user request for an airline reservation ("a query/answer routine") (column 9, 
lines 5 to 30). 



Application/Control Number: 10/684,357 Page 9 

Art Unit: 2626 

8. Claims 15, 57, 72, and 74 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Barclay et al. in view of Lennig et al., as applied to independent 
claims 1 1 and 53, and further in view of White et al. ('272). 

Concerning independent claims 15 and 57, Barclay et al. in view of Lennig et al. 
disclose all the limitations of independent claims 1 1 and 53, but omit "wherein said 
second processing routine is configured with an amount of resources by said server 
computing system based on a bandwidth and transmission speed associated with the 
transmission link between said server computing system and said client system so that 
said second processing routine performs accurate recognition of said one or more 
words with a first latency that is less than a second latency that would result if said one 
or more words were recognized by said first signal processing routine and then 
transmitted over said transmission link." However, White etal. ('272) teaches a 
distributed voice user interface, where a number of factors are considered in dividing 
speech recognition functionality between local devices 14 and remote system 12. 
These factors include an amount of processing and memory capability available at each 
of local devices 14 and remote system 12, the bandwidth of the link between each local 
device 14 and remote system 12, and the kinds of commands expected from the user. 
(Column 5, Lines 23 to 37) Implicitly, by optimizing division of functionalities between a 
local device and a remote system, distributed speech recognition is faster, or has lower 
latencies, than would be present without the optimization ("with a latency that is less 
than a second latency"). An advantage is to provide a bulk of hardware and/or software 
for implementing a sophisticated voice user interface at a single remote system to 
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substantially reduce costs. (Column 2, Line 65 to Column 3, Line 14) It would have 
been obvious to one having ordinary skill in the art to configure processing routines 
between a client system and a server system based on bandwidth and transmission 
speed to reduce latency as taught by White etal. ('272) in a client/server speech 
recognizer of Barclay et al. for a purpose of obtaining a reduction in cost for a 
sophisticated voice user interface. 

Concerning claims 72 and 74, White etal. ('272) teaches that each local device 
14 can be a relatively small, portable, inexpensive, and/or low power-consuming "smart 
device", such as a personal digital assistant (PDA) or a smart telephone (column 5, 
lines 38 to 47); dividing speech recognition functionality between local devices 14 and 
remote system 12 is based on an amount of processing and memory capability 
available at each of local devices 14 (column 5, lines 23 to 37). 

9. Claims 14 and 56 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Barclay et al. in view of Lennig et al. as applied to claims 1 1 and 53, and further in 
view of White et al. ('272) and Huang et al. 

White et al. ('272) discloses evaluating computing resources available at a local 
device 14 ("said client system") and a remote system 12 ("said server"). (Column 5, 
Lines 23 to 37) Lennig et al. teaches that mel frequency cepstra can include dynamic 
static parameters, C, and dynamic parameters, AC. (Column 4, Line 52 to Column 5, 
Line 28) However, Lennig et al. does not expressly disclose delta and acceleration 
coefficients for MFCC's, although it is well known that dynamic parameters include delta 
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and acceleration coefficients for MFCC's. Specifically, Huang et al. teaches LPC 
cepstra analysis, where 12 mel-frequency cepstral coefficients, 12 delta mel frequency 
cepstral coefficients, and 12 delta delta mel-frequency cepstral coefficients are 
employed to model the speech signal. (Column 5, Lines 6 to 20) An objective is to 
provide feature extraction for spectral analysis of speech to represent a portion of an 
utterance. (Column 3, Lines 25 to 43) It would have been obvious to one having 
ordinary skill in the art to employ a representation of mel-frequency cepstral coefficients 
("MFCC's") with delta and delta delta ("acceleration") coefficients as taught by Huang et 
al. in a speech recognition method of Lennig et al. for a purpose of representing a 
portion of an utterance by a known method of feature extraction for spectral analysis of 
speech. 



Response to Arguments 

10. Applicants' arguments filed 21 April 2009 have been fully considered but they are 
not persuasive. 

Firstly, Applicants maintain that the rejections under 35 U.S.C. §112,1 st If, of 
claims 14, 56, 72, and 74 as presenting new matter are improper. Applicants argue that 
the limitation of "on a connection by connection basis" is supported by the originally-filed 
Specification. Applicants aver that the "initialization" feature discussed in the rejection is 
irrelevant, and say, "All the claim states is that there is an evaluation of computing 
resources available at the client system and the server computing system." 



Application/Control Number: 10/684,357 Page 12 

Art Unit: 2626 

However, it is respectfully submitted that the whole point is that the claims say 
more than simply that there is an evaluation of computing resources available at the 
client system and the server computing system; the claims go further, and say that the 
evaluation is on "a connection by connection basis". If that were all that the claims said, 
then Applicants should have no problem cancelling the limitation of "on a connection by 
connection basis". 

Moreover, even assuming that the discussion of the "initialization" feature is 
irrelevant, Applicants still have not established that they have support for the limitation 
of "on a connection by connection basis" in their originally-filed Specification. Generally, 
once a rejection establishes a reasonable basis for challenging the claims as setting 
forth new matter, the burden shifts to Applicants to show that the limitation is indeed set 
forth by the originally-filed Specification. Normally, this is relatively a straightforward 
matter and does not present a problem: Applicants are typically agreeable in 
amending/cancelling the non-supported subject matter if it is not present in the 
Specification, or pointing out where the subject matter is clearly set forth if it is indeed 
present. Here, however, Applicants have continued to assert that the claimed subject 
matter is present even though it is pretty clear at this point that it is not disclosed. 

Applicants go on to contend that there is "nothing irreconcilable" about the claim 
language and the Specification, saying that SR engine is not configured until it is 
determined what the capabilities are of the client and the server, and stating, "Unless 
one always connects to the same server . . . with the same client this information cannot 
be ascertained without first establishing a connection between the two clients." 
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Respectfully, whether there is anything "irreconcilable" about the claims and the 
Specification is not the standard for resolving a rejection for new matter. Applicants are 
once again assuming facts that are not disclosed, and are not pointing to where this is, 
in fact, disclosed by the originally-filed Specification. Even if the capabilities of the 
server and the client cannot be ascertained without first establishing a connection, that 
does not imply that the capabilities are ascertained "on a connection by connection 
basis". They could be ascertained many times within the same connection or on a daily 
schedule. 

Applicants argue that the limitation of the client device being a portable device 
and speech values being configured based on the processing abilities of the portable 
device is supported by the originally-filed Specification simply because examples of 
portable client devices are provided. It is agreed that the Specification does provide 
brief mentions in scattered places of examples of portable client devices, but does not 
make a link to establish that each portable client device has different processing 
abilities. Applicants' originally-filed Specification did not appreciate this feature at the 
time of filing, and it is respectfully maintained that Applicants are now, ten years later, 
trying to capture subject matter that was not originally disclosed. 

Secondly, Applicants traverse the rejection under 35 U.S.C. §1 03(a) over Barclay 
et al. in view of Lennig et al. Applicants argue that the rejection has "strangely changed 
course", and runs afoul of the PTO rule that requires compact prosecution. Applicants 
say that, under the treatment of the case, prosecution will continue indefinitely. 
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However, Applicants should note that any "change in course" resulted from 
Applicants' defective claim for priority in the original filing of the current application. This 
required numerous petitions by Applicants on 31 December 2007, 12 May 2008, and 1 1 
September 2008, where at one point the current application was actually abandoned. 
The change in the rejection was a shift in reliance on He et al. to reliance on Lennig et 
al. as disclosing MFCC's because the former reference was no longer prior art following 
the eventual acceptance of Applicants' new effective filing date following the petition. 

Moreover, Applicants point about compact prosecution is not well taken. 
Certainly, compact prosecution is a desired goal in patent procedure, but it is always the 
nature of patent prosecution for both the examiner and the applicant to present 
arguments and interpretations of what the reference says, and there is frequently some 
back and forth in how the USPTO and applicants attempt to characterize the prior art. 
Even assuming everything that Applicants have said here, i.e., that "the Examiner has 
now strangely changed course", there is nothing procedurally improper about the 
rejection. He et al. may have better disclosed the features about the grammars, but, 
once He et al. was eliminated as prior art due to Applicants' amendment to establish a 
new priority date, Barclay et al. was found to disclose the features about the grammars, 
too. Nor does the Examiner wish in any way to prolong the prosecution indefinitely, 
either; if Applicants submit allowable claims, the Examiner will be more than happy to 
allow the Application. 
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Thirdly, Applicants contend that Barclay et al. fails to disclose the limitation of 
sending grammar related information to the server. Applicants say the rejection is 
based on a strained interpretation of Barclay et al. 

Substantively, however, Barclay etal. clearly discloses this feature at Column 8, 
Lines 23 to 28. Here, it is expressly stated that when the client is sending to the server, 
the client may specify what grammar the speech processor should use to recognize the 
current speech input or provide extra arguments over and above the speech inputs. 

Thus, Applicants' arguments that Barclay et al. only discloses grammars that are 
static for a weather page, whereas Applicants' claims might use one grammar for 
weather in Boston and another grammar for weather in Dallas, is not well founded. 
Applicants are again attempting to introduce extrinsic evidence of features that are not 
set forth by the claims, and that are not disclosed by the originally-filed Specification. 
Although the claims are interpreted in light of the specification, limitations from the 
specification are not read into the claims. See In re Van Geuns, 988 F.2d 1 181 , 26 
USPQ2d 1057 (Fed.Cir. 1993) 

Fourthly, Applicants argue that claims 71 and 73 are allowable because 
Barclay et al. discloses a LISTEN button is "clicked" by a mouse, and not "pressed" as 
set forth by the claims. 

However, it is maintained that clicking a button on a mouse is fully equivalent to 
pressing a button, as claimed. Nor is there anything in the claims or Applicants' 
originally-filed Specification that limits to a non-mouse button. 
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Finally, Applicants traverse the rejection under 35 U.S.C. §1 03(a) of claims 15, 
57, 72, and 74, further in view of White et al. Applicants present arguments as to the 
nature of what constitutes inherency, and say that White et al. does not mention dividing 
the speech recognition functionality in a manner that reduces latency based on the 
conditions in the link. 

Applicants' definition of what constitutes inherency is well taken, and it is agreed 
that the rejection is using the terms "implicitly" and "inherently" interchangeably. 
Factually, though, it is not always necessary that the prior art disclose the limitation in 
haec verba. Broadly, "latency" in communications simply refers to delays, either in 
processing or in transmission, and that is what is taught by White et al. The bandwidth 
of the link is expressly considered in dividing functionality by White et al. Moreover, the 
whole point of distributed processing is to reduce latency. This is expressly stated by 
Barclay et al. at Column 7, Lines 31 to 32, where it says that the purpose of sending 
quantized features to the server instead of simply performing the recognition at the 
client "is to reduce the latency time". Similarly, Barclay et al. notes an improvement 
over the prior art SLAM developed by OGI for a bandwidth limitation having excessive 
time delays when features are not extracted at the client before transmission at Column 
3, Lines 31 to 46. 

Therefore, the rejections of claims 14, 56, 72, and 74 under 35 U.S.C. §112, 1 st 
as failing to comply with the written description requirement; of claims 11 to 13, 53 to 
55, 71, and 73 under 35 U.S.C. §1 03(a) as being unpatentable over Barclay etal. in 
view of Lennig et al.; of claims 15, 57, 72, and 74 under 35 U.S.C. §1 03(a) as being 
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unpatentable over Barclay et al. in view of Lennig et ai, and further in view of White et 
al. ('272); and of claims 14 and 56 under 35 U.S.C. §1 03(a) as being unpatentable over 
Barclay et al. in view of Lennig et al., and further in view of White et al. ('272) and 
Huang et al., are proper. 

Conclusion 

1 1 . THIS ACTION IS MADE FINAL. Applicants are reminded of the extension of 
time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to MARTIN LERNER whose telephone number is 
(571 )272-7608. The examiner can normally be reached on 8:30 AM to 6:00 PM 
Monday to Thursday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, David R. Hudspeth can be reached on (571) 272-7843. The fax phone 
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number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Martin Lerner/ 
Primary Examiner 
Art Unit 2626 
May 22, 2009 



